A Position-Aware Transformer for Image Captioning

Image captioning aims to generate a corresponding description of an image. In recent years, neural encoder-decoder models have been the dominant approaches, in which Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM) are used translate image into natural language description. Among these visual attention mechanisms widely enable deeper understanding through fine-grained analysis even multiple steps reasoning. However, most conventional based on high-level features, ignoring effects other giving insufficient consideration relative positions between features. this work, we propose Position-Aware Transformer model with image-feature position-aware for above problems. The firstly extracts multi-level features by using Feature Pyramid (FPN), then utilizes scaled-dot-product fuse enables our detect objects different scales more effectively without increasing parameters. mechanism, obtained at first, afterwards incorporated original captions accurately. Experiments carried out MSCOCO dataset approach achieves competitive BLEU-4, METEOR, ROUGE-L, CIDEr scores compared some state-of-the-art demonstrating effectiveness approach.